In light of recent events regarding the performance of the Windows terminal 1, I thought it would be worth spending some time on this topic, since it uncovers some of the problems that are pestering the software industry.
What happened was a typical internet soap opera televised on Github. An experienced programmer posted a bug report about the slow text rendering in the terminal and after some back and forths with the maintainers, one of them came up with the following statement:
I believe what you’re doing is describing something that might be considered an entire doctoral research project in performant terminal emulation as “extremely simple” somewhat combatively…
A few hours later another programmer came up with the prototype of a much faster terminal renderer, proving that for an experienced programmer a terminal renderer is a fun weekend project and far away from being a multiyear long research undertaking. As embarrassing as this is for Microsoft, which despite owning the platform and its APIs still struggle to produce a reasonably performing software, I think part of the problem is simply caused by the corporate incentives.
Based on my limited anecdotal evidence, it seems to me that the software industry doesn’t value technical excellence as much as it values having warm bodies report to you. This in turn limits the amount of experience an average programmer working on these big projects will have, just because they are incentivized to leave the hands on implementation work for some sort of a leadership role as soon as possible.
Yet, large enterprise projects expand at a faster rate than the universe itself and somebody still has to program all those features. But, who is going to do all that work?
A programmer’s lifecycle
In order to be able to answer on the previous question we have to understand the lifecycle of a professional programmer. I think programmers roughly fall into 3 categories:
-
Wet behind the ears: the ones that have just started working as a professional programmers and need lots of guidance.
-
Sort of knows what they are doing: professional programmers, who have sort of figured what is going on and can mostly finish their tasks on their own.
-
Experienced: the ones who have been around the block for a while and are most likely the brains behind all the major design decisions of a large software project. Coworkers usually turn to them when they need advice for a hard technical problem. Some programmers will never reach this level, despite their official title stating otherwise (but more on that later).
Obviously, this ranking is just a rough generalization and we could nitpick about these categories till the end of time, but for our little discussion it’s a good enough approximation. I have specifically avoided putting years of experience on these categories, since I believe your self interest and curiosity matter more than the years spent writing getters and setters according to some corporate style guide.
When you join the working world as a professional programmer, you will most likely end up in the lowest rank - the so called “Wet behind the ears” category. You will probably be assigned a mentor who will look over your work and make sure that you perform your work according to the company’s standards and regulations. You won’t have much say in what you are doing and you will be handed all sorts of tasks; both interesting as well as less interesting tasks that nobody wanted to touch 2.
After a few years of sweeping the floor, kicking ass and chewing bubble gum in the “Wet behind the ears” level, your boss will eventually realize that you have proven yourself enough times and you are ready for more. The fancy title of “Sort of knows what they are doing” programmer is yours.
Together with increased salary also comes additional responsibilities. At this point you will be expected to work more or less independently without someone having to watch every decision that you make. Maybe you may have to design a smaller part of the project. Maybe you will have to take over the tasks of a more experienced developer when they are on their well deserved vacation. Maybe you will have to check the output of someone in the lower ranks. Maybe you will have to talk with the customer in order to resolve their issues. This field is broad, who knows?
The “Sort of knows what they are doing” rank is also the stage where the wheat is separated from the chaff. The pointy haired bosses are always on the lookout for someone capable who may be able to take over their current work. Now that you have proved yourself multiple times, it’s time to think about the future and “grow” into a more responsible role. What do you say about leading the slightly bigger feature with all these “Wet between the ears” programmers under you?
After all, it’s the obvious next step for you. When the company is growing and adding people to its roster, who is going to lead them? Obviously not the new and inexperienced people and obviously not the grumpy experienced people who have already turned down such organizational role in the past.
If you decide to go down that path, you may think that nothing is going to change: you will still be doing your usual job, except now you will have to spend some time for coordinating other people. Sometimes this small chunk of time dedicated for coordination escalates into full blown time coordination and suddenly you are no longer doing any of the work that you were previously good at.
As a consequence of that, your own knowledge also stops at the “Sorts of knows what they are doing” level, since you don’t get better at things that you are not doing on a daily basis. Artists seem to be well aware of this problem; they know that you have to study the old masters and put in the hours to become the master. On the other hand, software developers often equate years of experience as some sort of a badge of knowledge. Sitting in an office and shuffling papers around for a decade doesn’t make you a master programmer. Writing software does.
Am I condemned to stay in the “Sort of knows what they are doing” level forever if I decide to pursue the management path? No, some are also stubborn enough to make it into “Experienced” programmer’s category, but there are few and far in between who are willing to put in that amount of effort on top of their coordination work.
As shown in the made up chart above 3, the higher up the tree you climb the lonelier it gets. At last you reach the “Experienced” programmer category, where your responsibilities and political power increase once again. Usually you will be expected to come up with detailed designs, task estimates, resolving hard bugs, code reviews, interviewing job candidates, improving internal processes, mentoring and playing office politics.
Since you have all these other things to worry about, you can’t spend nearly as much time programming as you used to when you were a member of the lower ranks; you are forced to learn how to delegate the hands on work to the rest of the team. After all, one person alone can only achieve so much, however great they may be.
So, who is developing all the features then?
Due to the way the programmer’s ranking system works, an average company will have a big crowd of inexperienced people and very few who can tackle complex problems. For this reason, the vast majority of software is written by the ones who are still learning the ropes.
How much do “Wet behind the ears” programmers really know? Not a whole lot and that shows up in the quality of our software. Sure, the experienced programmers will take care of the design problems, but that is not going to save your bacon if the implementation of a great design is terrible.
If you want to produce a high quality software, you also need enough experienced people in the trenches that are not merely delegating the orders from the ivory tower. How many times have you encountered a big ball of complicated code that could be drastically simplified if only the developers knew their development environment better? There is more than one way to skin a cat, but some approaches are just better than others.
But, wait! How do you know what works and what doesn’t? Well, you sort of have to build the intuition and the only way to build it is by programming a lot. You can’t really learn that by reading random articles, since you have to get your hands dirty before you internalize the lessons that you have read about.
Notes
-
The boring tasks will be often presented as a challenge to prove yourself, but most of the time you are really just sweeping the floor; someone has to, and the newcomer is usually the most appropriate person for that. ↩︎
-
Nobody knows what the real numbers look like, but you can sort of judge based on what you see around you. Especially, if you are an interviewer you directly experience how the job market looks like (although it also depends on how glamorous your company is. Well known companies will attract better people).
Some say that the majority of professional programmers have less than 5 years of experience, but there is no way to verify that claim. ↩︎